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Executive 
Summary 



The basic premise for this paper is that any 
educational program or technology should be 
evaluated on the basis of the theoretical foun- 
dations or perspectives on which it is 
implicitly or explicitly based. Phase I of B.C/s 
SITP had a fairly general rationale: the desire 
to link districts and students to each other and 
to various sources of information, including 
legal experts, writing mentors and informa- 
tion databases. 

For Phase II, formulating a more in-depth 
and specific rationale would assist in identify- 
ing research topics and methods of evaluation 
that would focus on the impact of telecommu- 
nications technology on student learning in 
the context of the teaching and learning envi- 
ronment. 

This paper reviews recent literature in areas 
related to the many aspects that influence 
learning through telecommunications and 
computer conferencing, and is stnictured in 
three main sections. 



The first section is an overview of how cogni- 
tive theory can provide a basis for 
understanding computer-mediated communi- 
cation (CMC) activities, with respect both to 
their educational potential and to the interac- 
tion of the learner with a computer system. 
There is a large body of literature relating to 
these topics, all of it related to attempts to 
understand how the individual interacts with 
information on the basis of previously 
acquired knowledge and /or mental models. 

The second section reviews the literatvire that 
relates technologies such as CMC to their 
impact on education and society. This pro- 
vides a basis for thinking of new roles for 
teachers and leame^^ and can also help us 
identify some of the prospects that students 
will face as they reach the workplace. 

The third section describes research theories 
and methods that have been used in similar 
contexts and which might assist us in finding 
ways to evaluate the current project. 
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Introduction 



It is difficult to evaluate the effectiveness of 
any new educational program or technology 
in the absence of a theory or rations le behind 
its initial implementation. For inst^.nce, Pea 
and Soloway, in a report for the U.S. Congress 
Office of Technology Assessment (1987), 
describe the importance of research in the cog- 
nitive, social, and instructional sciences, and 
the accompanying use of theory-guided edu- 
cational technologies, as a tool to help bridge 
the "ever-widening gaps between schools and 
society" (pp. 33-34). Placing their research 
agenda in the context of an educational sys- 
tem which is intended to relate to the needs of 
society, provides them with a set of criteria for 
the evaluation of that research. In the same 
report, these authors also state that theories of 
technology in education can develop respon- 
sivcly to the feedback provided by 
applications in real settings (p. 47). 

The literature reviewed in this paper is orga- 
nized to assist the reader in identifying 



current thinking about some of the main 
issues that have impact on the research agen- 
da for Phase 11 of SITP. Research in cognition 
can provide a basis for understanding strate- 
gies of learning, both as a background for the 
creation of educationally relevant telecommu- 
nication activities and with respect to the 
system itself and the user interface. Research 
relating to cultural (including school culture) 
and other changes that accompany the 
expanding access to information and associat- 
ed technologies can help us address the new 
roles of teachers and learners, and can also 
help us understand the place of technology in 
the society of the future. Finally, an overview 
of the research methods and results from 
other technology-based projects (both within 
and outside of educational environments) can 
assist us in finding methods to evaluate our 
current project. 
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The Cognitive 
Perspective 



To an increasing degree over the past two or 
three decades, cognitive science has provided 
a perspective for understanding the interac- 
tion of an individual's mental functioning and 
the complex contexts in which it occurs. Tliis 
perspective has influenced fields such as psy- 
chology, economics, ergonomics, linguistics, 
artificial intelligence, information sciences, 
and - of particular importance to this litera- 
ture review ~ education. 



What does cognitive science tell us about 
learning? 

The attempt to define learning is frequently 
omitted from discussions (whether verbal or 
written) about learning and effective learning 
environments. At this point, it might be suffi- 
cient to say that the implied (or sometimes 
actually stated) definition of learning in most 
of the literature reviewed here includes the 
notion of cognitive change. 

Some main points from the cognitive perspec- 
tive are as follows: 

- learning involves the active construction of 
mental models 

- the learner's world view (existing models or 
structures, including strategies for catego- 
rizing and integrating information and 
concepts) mediates the processing of new 
information 

- learning can never be completely detached 
from context. 

The cognitive view of learning accommodates 
what we intuitively know about human diver- 
sity and adaptability. Chesebro & Bonsall 
(1989) have described this in the following 
way: "Human beings react to many different 
kinds of information, develop different con- 



ceptual systems to deal with them, and ulti- 
mately generate multiple ways of perceiving 
and understanding any single event or phe- 
nomenon" (p. 39). Active learning (including 
strategy use), the transferability or generaliz- 
ability of learning, and the importance of the 
context for learning are interrelated areas that 
have received much attention in cognitive 
research. This paper can only touch upon a 
few of the many recent examples of writing 
related to these important ideas. 

An adept learner makes use of cognitive 
strategies for learning and makes use of 
metacognitive strategies to monitor his or her 
progress (Pressley, Goodchild, Fleet, 
Zajchowski, & Evans, 1989, Paris, 1988). A 
strategy is "an ordered sequence of mental 
steps enacted to satisfy a goal" (Lehrer, 1989, 
p. 303). In order to learn adeptly, the learner 
must first know some strategies, but - of 
equal or greater importance - he or she must 
also know how and when to apply them. 

The investigation of strategy use is now a 
major topic in cognitive research. Can strate- 
gies be taught? (Pressley, Forrost-Pressley, 
Elliot-Faust, & Miller, 1985). Why are people 
not strategic when they need to be? 
(Rothkopf, 1988). How can strategies be 
taught to promote their generalization or 
transfer to different situations? (Perkins & 
Saloman, 1988, 1989). What is the importance 
of context? (G^.;. \er, 1990). What is the role of 
basic knowledge? (Hirsch, 1987), Can strategy 
use or general problem solving ability be 
enhanced by computer use? (Lehrer, 1989, 
McCoy, 1991, Papert, 1980). 

At this point in time there is not a clear con- 
sensus regarding the importance of factual 
content (how much content? what content?) in 
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relation to strategy use and the generalizabili- 
ty of learning. The issue has become 
somewhat polarized and is regarded by some 
as rc-quiring the choice between cultural litera- 
cy and critical thinking. For ertample, Hirsch 

(1987) views the basic knowledge that makes 
one culturally literate as something that 
becomes lost if too much time is spent teach- 
ing critical thinking; he is pessimistic about 
the transfer of learning strategies and general 
skills. An alternative viewpoint, as expressed 
by Salomon (1988) and Terkins and Salomon, 

(1988) is to view both generalizable learning 
skills and 'local' knowledge as being comple- 
mentary, and not in competition with each 
other. Pea and Solo way (1987) hold a similar 
optimistic view about the relationship 
between generalizable learning and factual 
knowledge: 

The uecessit}/ of subject matter bwxdedgc in 
expertise has heeii recognized for centuries. 
What is neio is the research-based recognition 
that it is not a knowledge base of facts per se 
that should bean instructiojml goal Instead, 
students need to acquire facts, principles, or the- 
ories as ccnicefytual tools for reasoning and 
problem solving that they can see makes sense 
because they have consequotces in meaningful 
ccnitexts (p. 37). 

Where one stands on this issue clearly influ- 
ences the types of learning activities that will 
be deemed appropriate for effective learning. 

In summary, the cognitive viewpoint places a 
research and implementation emphasis on the 
ways in which an individual can learn how to 
become a good learner. Context and general- 
izability of learning are both important factors 
for consideration. "Learning becomes the 
grinding of better (conceptual) lenses that 
help us make more sense of the world." (J.S. 
Brown, 1989, p. 9). 



What does cognitive science tell us about 
school? 

School is certainly one of the most important 
environments in which cognitive change (or 
learning) is expected to take place. What can 
be done in school to promote the learner in his 
or her quest to make more sense of the world? 



Particularly as technology now has the poten- 
tial to provide the learner with access to 
increasing amounts of information, it is 
important for school-initiated activities to be 
arranged in such a way that learners can use 
that information effectively. How do learners 
organize, structure, interpret, evaluate and 
understand the information they will be 
exp(3Sed to in school? How can learning envi- 
ronments (including the people who are part 
of them) assist with the processes of learning? 
Schools must respond to many questions 
posed by a changing world. Those responses 
should be based on the most current knowl- 
edge about the learning processes. 

A basic question relating the cognitive per- 
spective to the role of learners within a 
teaching environment has been addressed by 
Phil Winne (1985): "The emphasis in the cog- 
nitive mediational view of teaching is not on 
teacher behaviours but rather on students' 
cognitive interactions with teaching" (p. 673). 
This quotation refers to the human teacher, 
but books, video, computer programs, infor- 
mation systems and other media can also 
fulfil a 'te.:v:hinR' role. The cognitive view con- 
siders effective teaching to be a response to 
the learning requirements of the individual. 
The fcKus is on the learner. 

What are some implications of the cognitive 
viewpoint for educators and learners? 

- the learner's present level of understanding 
is the starting point 

- the learner often has alternative conceptual 
frameworks which frequently interfere with 
the learning of the desired framework 

- prior knowledge should be activated as 
much as possible 

- learners need to be aware of relationships 
among and between concepts and principles 
in content 

- learners need to be involved and engaged as 
active participants in the learning process 

- learners need to be aware of the effective- 
ness of their learning strategies. 

What can be done to actually implement a 
learning environment that is consistent with 
the cognitive viewpoint? 

Brown, Collins and Duguid (1988, 1989) dis- 
cuss authentic versus school activity and 
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describe a "cognitive apprenticeship" as a 
powerful way to conceive of learning activi- 
ties. "Apprenticeship" implies the importance 
of modeling, coaching and 'fading' (or gi adu- 
al withdrawal of the coaching as the learner 
gains competency) through authentic activit>' 
as well as the importance of social interaction 
and collaboration. The learner observes and 
gradually participates fully. 

"Knowing (and not just learning)... is inextri- 
cably situated in the physical and social 
context of its acquisition and use. It cannot be 
extracted from these without being irretriev- 
ably transformed" (Brown et al., 1989, p.l). 
Lewis (1990) explains that a number of cogni- 
uv..- ."wvv a theory of 

social interaction which incorporates the 
effects of cognition should be developed and 
would replace existing theories of cognition 
wliich attempt to incorporate the effects of 
social interaction. The culture of school, as a 
social environment, is thus extremely impor- 
tant. As the school environment deviates 
further and further from the tyjx'b of environ- 
ments where students will ultin-wtely use their 
learning, the more likely it becomes that stu- 
dents will be unable to put their school 
learning to use. 

Real world knowledge is never static, 
although curricula and learning resources 
sometimes are: 

Within a culture, ideas are exchanged and modi- 
fied and belief systems deiteloped and 
appropriated in part through conversation and 
narratives, so these must be promoted, not inhib- 
ited. Though these are oftm anatham to 
traditional schooling, they are an essential com- 
poncfit of social interaction and thus of learning. 
(Broxvnet aL,1989, p. 26.) 

These notions challenge much of what hap- 
pens within the traditional school culture, 
where infom^ation is presented in an abstract- 
ed format and used in a way that is usually 
quite unlike the way it is used by practition- 
ers: "an unacknowledged dependency on 
such school-based cues makes the learning 
extremely fragile" (p. 14). "Authenticity in 
activity is paramount for learning" (p. 15). 

Note that the cognitive perspective as 



described by Brown is considered to be an 
cxtrenrH? iw by some other cognitive scien- 
tists such as Merrill, Li, and Jones (1990) and 
Nickerson (1988) who prefer a conservative 
construct! vist view as described in the follow- 
ing statement: "learning is best described not 
as a process of assimilating knowledge but as 
one of constructing mental models, The learn- 
er's role is seen as necessarily an active one" 
(Nickerson, 1988, p. 5). In this view, however, 
active leaniing can involve more abstracted 
formats of knowledge and can (although it 
does not necessarily have to) take place in a 
less socially-oriented environment. 

From both the conservative and more extreme 
cognifi""* .;pr)infrs, the learner is no longer 
viewed as a tabula rasa. For this reason, cogni- 
tive theorists consider it desirable for the role 
of the teacher to move away from that of 
information transmitter. Tlie next few para- 
graphs will briefly describe a small sample of 
similar cognitive perspectives relating to the 
school environment. 

Newman, Griffin and Cole, have entitled their 
1989 book The Construction Zone: Working for 
Cognitive Change in School. The premise of this 
book is that school can be a 'construction 
zone,' or place for the meeting of minds, 
where "complex shared activity . . . allows one 
mind to appropriate another's thinking and 
that provokes new meanings" (p. xiv). It is 
important to note here that the "mind" being 
met by the learner could be a teacher, a IxH)k, 
another child, etc., and that the "meeting" 
couldTx? face-to-face or through media such 
as telecommunications. They describe how 
the consideration of the interaction of social 
and individual processes leading to cognitive 
changes can have a profound impact on how 
the teacher makes decisions, when the teacher 
intervenes, and the ty^^e of evaluation and 
assessment that takes place in the classroom. 
Th'^ social construction of knowledge is a 
model that allows for a degree of creative 
change that is not accommodated in the 
knowledge transmission model of education. 

Gamer (1990) discusses the inappropriateness 
of the traditional classroom situation in pro- 
moting cognitive and metacognitive strategy 
use. Because the performance orientation of 
many such classrooms rewards a type of suc- 
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cess that can be defined as outperforming oth- 
ers with the least possible effort, students 
develop superficial strategies that may be 
immediately effective but they are not the 
type of strategies that really relate to the pro- 
cess of learning. This is espx^cially important 
to consider when attempting to encourage 
learners to be self-directed or to transfer their 
leal ning to new but related situations. The 
competitive classroom is thus a barrier to 
learners' motivation in constructing, refining 
and using strategies: "without high self- 
esteem and the tendency to attribute success 
and failure to their level of effort, both chil- 
dren and adults are unlikely to initiate or 
persist at strategic activity" (p. 521). 

Pea and Soloway, (1987), in a report prepared 
for the U.S. Office of Technology Assessment, 
summarize what has been learned from cogni- 
tive research. They state compelling evidence 
of a need to move away from the knowledge 
transmission model of education and revital- 
ize the deeply-troubled .school system. Tliey 
conclude that in the present system, students 
are unable to utilize their knowledge for rea- 
soning and problem solving and they are also 
unable to take advantage of the potential 
offered by technology' in support of learning, 
reasoning, communicating and collaborating. 

From tlie perspective of cognitive science, we 
have now defined the ideal learning situation 
as one where the learner can function in a self- 
motivated manner to actively acquire and use 
information from all kinds of sources, where 
the learner is encouraged to apply strategies 
and monitor them for their effectiveness, and 
where the learner works to construct individ- 
ually relevant meaning. A powerful context 
for the construction of meaning involves the 
authentic culture and social interactions of a 
cognitive apprenticeship. The information- 
rich world of telecommunications seems to 
have the potential to be an appropriate envi- 
ronment for both acquisition of information 
and social interactions (Teles, 1991). What are 
some of the important considerations? 



Information Overload 

Earlier in this paper there was a description of 
what cognitive science tells us about learning. 



A central concept is the learner's mediation or 
the interaction and integration of new infor- 
mation with a pre-existing world view. 
Mediation implies selective attention, which is 
not a new concept. We know intuitively that 
we are usually able to screen incoming but 
unwanted inforn-^ation, such as the back- 
ground noise in a room when we are trying to 
hold a conversation. Mediation and selective 
attention in learning, however, are gaining 
importance as a topic of study, as more and 
more information is becoming available in the 
learning environment. Selective attention is 
required to avoid cognitive overload. What is 
the important information to select? How can 
learners acquire the judgment to effectively 
select information? 

Relan, (1991) describes how in an "open- 
ended environment, decision-making may 
contribute to an unforeseen cognitive over- 
load" (p. 12) but goes on to state that "placing 
vast amounts of networked information and 
open-ended tools at the learners' disposal 
encourages thern to generate novel solutions 
to criterion tasks, enhancing their creativity 
and problem-solving tasks" (p. 13) explaining 
that "devices which monitor learners' actions 
should be constantly available" (p. 13). Tlie 
'devices' refer to feedback mechanisms, strate- 
gy guides and navigational help. "It would be 
the instructional designer's chore to make 
learning more manageable, especially to 
novice users of the environment" (p. 13). 

O'Shaughnessy, Coskuntuna, and Kantro 
(1987) concluded from experimental evidence 
gathered in an information environment that 
"people continually circumvent the overload 
problem by developing strategies to selective- 
ly choose the information they need and 
ignore other information" (p. 61). The impor- 
tance of strategy use has been addressed 
earlier in this pajx?r, but it should be re- 
emphasized in relation to the cognitive 
overload of information systems. All too 
often, learners misuse or neglect to use strate- 
gies, even when they know them. 
Telecommunications and information systems 
are very new to our society and to our educa- 
tion system; therefore effective strategies (for 
instance, in hypermedia use) are not fully 
developed. (liontext again appears as a major 
issue. These authors state that there is a 
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"problem for people who are gathering infor- 
mation they are not familiar with: they 
consider a large number of attributes to be 
important, but they are limited in the amount 
of attributes they can consider" (p. 66). Tliis 
has a definite implication for the structuring 
of on-line information retrieval and decision 
support systems, 

DiScssa (1988) discusses a misleading aspect 
of the information explosion: "The mistaken 
attitude is that all it takes is a mastery of a 
properly organized collection of facts. 
Instead, learning is better viewed as conceptu- 
al change" (p. 47). In a similar vein, Pea and 
Soloway (1987) state that 

. . . informafion access does not make education. 
Students need to fontiulate inquiries that infor- 
mation search may play a role in and that to 
hmohoxo to filter and organize the infonnation 
they obtain through searches in order to address 
the questions that led them to their initial search. 
Students xoill need special hroxosing tools for 
examining large infonnation databases designed 
to take account of features of their understand- 
ing and of learning etwironments. Cognitive 
and instructional research . . . needs to be 
extended so that xvc knoio more hoiv students 
can learn to synthesize information from diverse 
sources collected for a variety of purposes. Haio 
should searches and inquiries be formulated? 
What are the information age equivalents to 
xvhat used to be taught as library skills? (p. 58). 

As in the Phase I SITP interim report (where 
teachers have requested structures to help 
them with classroom management and curric- 
ular links in the context of a rich 
information-access environment), there is a 
need expressed by teachers to find ways of 
helping learners interpret, evaluate and use 
information through effective and meaningful 
activities. 



Cognitive science, technology and education 

The need for devising appropriate activities to 
meet the cognitive requirements for learning 
in a technological environment seems obvi- 
ous. In the literature, there are a few 
examples where the technology has been used 
within a deliberate theoretical framework. 



The first three of the following examples focus 
on technology as a way of presenting informa- 
tion and helping individual learners make 
best use of it. The remaining examples deal 
more with environments where collaborative 
learning is encouraged through the use of 
technology. 



Cognitive theory as a basis for using technol- 
ogy in information presentation 

Smaldino and Tliompson (1990) discuss 
Gagne's nine events of instiuction (Gagne & 
Briggs, 1979) as a framework for structuring 
the use of computer technology (in this case, 
software used in a science unit). Various 
types of software (such as simulations and 
tutorials) and hardware (such as LCD projec- 
tors) arc described as appropriate to die 
following steps of learning: 

-gaining attention 

- informing the learner of the objective 

- stimulating recall of earlier learning 

- presenting new material 

- providing learning guidance 

- eliciting performance 

- providing feedback about performance 

- assessment of performance 

- enhancing retention and transfer. 

The above is very teacher-directed, although 
similar guidelines could be developed for a 
I. '.ore learner-focused experience. 

Kozma (1987) describes learning tools in rela- 
tion to what is known about the limitations of 
human cognition. The computer is seen as a 
cognitive tool which can amplify, extend or 
enhance human cognition. Activities are open- 
ended and learner-controlled within a system 
that defines or structures the work space such 
that learning-related skills and strategies are 
more likely to be activated (p. 21). The system 
"supplements working memory . . . 

- makes relevant, previously learned informa- 
tion available simultaneously with the 
acquisition of new information 

- promotes learner to structure, integrate and 
interconnect new ideas with previous ones" 
(p. 23). 

The STUDY system presented by Phil Winne 
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(in press) at the Literacy and Technology 
Symposium at the University of Victoria 
(1991) is another example of a technology- 
based learning environment based on an 
instructional theory. In this case, the theory 
casts the learner as artisan, cognitively pro- 
cessing knowledge to create and use 
knowledge structures. 



Collaborate, ve learning, cognitive theory and 
technology 

Mason (1988) describes computer conferenc- 
ing as a learning environment in tune with the 
objectives of self-directed learning, because 

- learning is a process ofexpandmg upon what 
the learner already btaws 

- the learner beccnnes an active seeker, respatisi- 
ble for fijtding out what he already bums and 
what he needs to learn 

- by fonnulati7ig what he knows and cotnmuni- 
cating it to his peers, the learner contributes 
both to his atun learning and to that of others 

- through discussion which graios from the com- 
mitment of the group, each individual engages 
his own desire to learn (p. 30). 

However, the reader is cautioned that 

. . . whett atte}npti7ig to displace the teacher from 
centre stage and hand over the responsibility for 
learyiing to the student, the innovator often finds 
not grateful thanks, but great resistance. For 
years, studmts have bem accustomed to authori- 
tarian teachers, to the systetn and an 
educatimml 7nilieu in which they accept the role 
of passive, dependent, co7npetitive learner. 
Consequefitly, they lack confidence in their mm 
judge^nent (p. 30X 

Mason considers the on-line environment to 
be appropriate to a theory of autonomous 
learning, because conferencing helps to "de- 
package knowledge" (p. 38), starting with 
what the learner knows and allowing for peer 
support. 

Scardamalia and Bereiter (1989) have 
designed a Computer Supported Intentional 
Learning Environment (CSILE ) on the basis 
of cognitive theory incorporating communal 
knowledge building and strategy use. CSILE 



is a networked computer environment in 
which students can share information and 
build collective databases. 

J.S. Brown (1985) discusses process versus 
product as an important cognitive context for 
using communal and informal electronic 
learning tools. One example is a set of tools 
for multiple authorship, which can keep track 
of all revisions of a written work and allow 
each author to comment and leave messages 
for the others. Another is a structure for com- 
munity information systems, so that 
specialized 'subcultures' of learning can 
develop across a broader geographical base of 
learners than is currently possible. "The right 
collection of tools should help the overall sys- 
tem to act as a mirror for the user's thought 
processes, enabling him to reflect on his ov^ 
thinking and improve his metacognitive 
skills" (p. 199). 

"One of the main differences heretofore 
between formal education and professional 
work has been that cooperative effort usually 
characterizes the workplace, while most stu- 
dents do solitary exercises, CMC opens up 
the possibility for a great deal more collabora- 
Hve work in educaHon." (Boyd, 1990, p. 274). 

Nipper (1989) describes how CMC has the 
potential to "overcome the problem of social 
distance between learners and teachers, not 
just geographical distance" and concludes that 
although there will be difficulties (for teachers 
in particular) because of the changing roles, 
"computer conferencing may contribute to 
less authoritarian concepts of learning and 
teaching" (p. 71). 



What else can the cognitive perspective 
tell us? 

Many other reports of technology in educa- 
tion describe activities that do promote 
strategy use, encourage collaboration and 
support cognitive change without specifically 
taking the cognitive theoretical perspective. A 
small number of these, specifically related to 
computer mediated communications, will be 
described later in this paper, in the section 
entitled A sample of applications of CMC. 
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So far, we have considered the cognitive per- 
spective in relation to fairly general 
educational activities, some of which incorpo- 
rate computer technology. Cognitive science 
also informs the more specific ways in which 
human beings interact with machines such as 
computers. Issues such as how the learner has 
constructed an underlying mental model of a 
computer system can influence whether or not 
that system is used effectively. A fairly large 
body of research, much of it gathered outside 
of the educational environment, exists in this 
area. Frye, Littman, and Soloway (1988) 
describe the situation as follows: "even 
though human-computer interaction is a topic 
of growing interest, that field has not focused 
on the special properties of educational soft- 
ware" (p. 451). The next section of this paper 
will describe some of the human-computer 
interaction (HCI) research. 



Cognitive science, information theory 
and human-computer issues 

The importance of the application of a cogni- 
tive perspective to human-computer and 
information science issues has been expressed 
by More (1990) "In a psychological sense we 
need to recognize the imjxirtance of human 
factors in designing, accepting and applying 
new systems" (p. 318), by Williges (1987) 
"Due to the complexity of the human, task, 
hardware, software, and environment config- 
urations, the designer must simultaneously 
consider a variety of factors that affect multi- 
ple dimensions of system performance" (p. 
67), and by many others. 

Such work has been seen as quite fruitful: 
"one of the unique features of such sugges- 
tions (from the cognitive view of information 
science) is that many of the ideas which have 
been couched initially in abstract or polemic 
terms have subsequently led to theoretical, 
experimental and practical advances in a 
broad range of information science activities" 
(Belkin, 1990, p. 11) or as an incomplete 
improvement over previous attempts: 

There are four basic reasons xohy systems are not 
working zvelh currait systems tetid to be 
desigrted in haste and without adequate develop- 
mental testing, curretit systems taid to he 



pushed through productiott and sales too rapid- 
ly, inadequate attention is given during design 
to the details of the human computa' interface, 
inadequate attention isgivett to operational test 
and evaluation systems before systetns are pre- 
sented to the public . ..it is understandable that 
excitement and anticipatiott sotnetiines replaces 
objectivity (Muckler, 1987, p. 4). 

In a review of Nigel Tucker's 1989 book. 
Interactive Media: The Human Issues, Ford states 
that "optimism must be tempered by the 
thought that the real prerequisites of signifi- 
cant advance are a knowledge of the sensitive 
interactions between human, cultural and 
technological aspects of learning which still 
elude us" (Ford, 1990, p. 343). 



Interface issues 

Reports from Phase I of the SITP have indicat- 
ed that tl^e current interface has, in fact, been a 
stumbling point for at least some of the partic- 
ipants. This section is an attempt to cover a 
small portion of the current thinking about 
appropriate directions in interface design. 

Graesscr, Gordon, Forsythe, and Greer (1991) 
define 'educational artifacts' as a category of 
devices designed to promote learning and 
describe the relation of cognitive theory to 
person-artifact interfaces, "A good interface 
would cater to the constraints of human cog- 
nition; the interface would minimize the time 
to perform tasks and the incidence of user 
errors'" (p. 16). Since the educational activities 
(perhaps a creative writing experience or for- 
mulating a question to ask an expert) 
involving telecommunications require a great 
deal of cognitive processing, it becomes even 
more important that the interface be as trans- 
parent as possible. The effort spent in 
correcting an error such as getting lost in the 
system will obviously take away from the 
effort spent in making thoughtful use of the 
information that can be acquired through the 
system. 

Rosson and Alpert (1990) state: 

In the solution of an interactive softivare design 
problm, an important cotnponent is the design- 
er's model of how the user will understand the 
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system . ..we refer to this as the desigit model, 
the user's actual understanding is termed the 
coficeptual model. The design model is com- 
prised of the entities, operatiatts, and relati-'us 
that ike designer expects (or hopes) the user xvill 
refer to when ximking xvith or, reasoning about 
the systan. Usability will be improved to the 
extent that the designer can create a design 
model that xvill lead to a coherent and compre- 
hensive conceptual model Researchers in 

user interface design have proposed a variety of 
techniques for building effective design models 
and evoking the correspoftding conceptual mod- 
els in users, (pp. 352-3). 

Such techniques include providing training 
manuals and advance organizers, creating 
interfaces that make use of analogies or 
metaphors and thus relate a system to some- 
thing already familiar to the user, sometimes 
even designing intentionally limited computer 
systems that mimic real-world systems and 
are therefore more intuitive to the user. These 
authors conclude that the latter is not really a 
desirable design technique because emulating 
a non-computer environment does not allow 
the designer to make use of the real power of 
the computer. 

The user 5 conceptual model describes his or 
her understanding of what the system can do, 
and how to move through the system (naviga- 
tion) to actually do those things. 

Nipper (1989) states: 

Pocrr navigatiofi tools, or lack of structural 
transparmcy, is a common constraint in com- 
puter confereftcing systems as we knozo them 
today... such constraints ingmeral may increase 
the problems of the ordinary dii^tarice learners, 
xvho have no prior experience of CMC, and 
therefore possess no tools or measi'res whereby 
they can organize the learning situution. The 
use of the syste^n may subsequently very 
unstructured and educationally ineffici-^t. 
(p. 68) 

Streitz (1987) considers human computer 
interaction to be a type of "interactive prob- 
lem solving" (p. 77) and describes how the 
problem solver must build up a mental repre- 
sentation of the problem based on both 
previous conceptual models and incoming 



information. In order to provide a system 
with appropriate information, both the 
designer's and the user's models must coin- 
cide, or be 'cognitiveiy compatiole.' In order 
to achieve this, the designer must ask, "What 
are the relevant properties of the interface 
which allow a novice user to build up a com- 
patible initial knowledge representation?" and 
"How can one provide transfer of learning 
from a mode suited to the novice to another 
mode for a user who has acquired expertise?" 
(p. 78). 

Streitz condnues to describe interfaces as 
being of two types: dialogue mode or 
metaphor world. Dialogue mode involves 
user-initiated commands while metaphor 
worlds often involve such options as menus 
and can be considered to be system-initiated. 
The former requires recall (to generate the 
exact command lines) while the latter requires 
only recognition (selecting the correct menu 
option). Metaphor worlds are therefore cogni- 
tiveiy much less demanding. An interesting 
result of this experimental study was that 
users selected the cognitiveiy more demand- 
ing dialogue mode as they gained expertise 
with the system. This phenomenon was also 
observed in Phase I of SITP, and seems (in 
both cases) to be related to a lack of power 
and flexibility in the particular menu-driven 
interfaces involved. 

A similar perspective is taken by Relan (1991) 
who describes the desktop interface (which 
fits the above definition of metaphor world) in 
relation to the need for a greater cognitive 
approach in computer-based instruction. 
Relan states that the desktop interface increas- 
es interactivity, allows greater learner control, 
relates to learning styles by incorporating 
both visuals and text, and allows learning 
strategies or cueing to be embedded in soft- 
ware design. 

These characteristics are described as not 
being necessarily desirable in all situations, 
and Relan suggests possible options: 

. . . learners should be given control, but if they 
do not make good use of it, the program should 
intervene and lead the student through instmc- 
tion . ..[or] . where studerits are provided 
with advisement on their current instructiatial 
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need, tvhich is iteratively updated statistically by 
the program . . .the option to act on the pre- 
scriptiojL is left to the studrnt. (p. 12) 

Another finding of Relan's (that might be 
related to Streitz's experimental results indi- 
cating increased expertise led to different 
choices in interface use) was that "for novice 
users, visuals are processed more quickly than 
text; however, this advantage is neutralized as 
users become more familiar with text" (p. 10). 

Belkin (1990) describes promising work from 
the field of information technology. Within 
the context of a library communication and 
information retrieval system, and applying a 
cognitive perspective, where the users' state 
of knowledge is the framework for computer 
architecture, he describes studies where "the 
cognitive viewpoint led to the specification of 
the functionality and architecture of intelli- 
gent computer-based information systems" (p. 
14). Here, designers based a system cn "gen- 
eral characteristics of librarian and user 
searching behaviour, which they could relate 
explicitly to the choice of mediating knowl- 
edge structure on the part of the librarian" (p. 
13). This information, gathered by a "thinking 
aloud" methodology, also incorporated a 
model of the user, a model of the actual docu- 
ments being sought and ultimately led to a 
relational model among librarian, user and 
documents. Here, user requirements were 
apparently structured in a way that could be 
understood by the designers, who could in 
turn create a system that incorporated the 
user's conceptual models. 

Hildreth (1989) discusses intelligent interfaces 
and intelligent retrieval systems, and states 
that interface issues cannot be handled sepa- 
rately from the internal workings of the 
system: "this author has seen too many bad 
query-language indexing and retrieval design 
implementations that could not be improved 
or glossed-over with any amount of 'front- 
end' software" (p. 51). User requirements are 
an important consideration: "A single search 
interface and a single retrieval model (e.g. 
Boolean) are not adequate for the variety of 
searchers and search needs that exist" (p. 105). 
and system intelligence is seen as being of two 
typ)es: delegation (or supplantation), where 
the system makes decisions based on pre-pro- 



grammed criteria, and augmentation (or con- 
sultation), where the machine assists the user 
but does not make its own decisions. State-of- 
the-art information retrieval systems include 
features such as natural language query input, 
linguistic analysis of input, graphic aids to 
browsing, probabilistic retrieval (weighted 
logic and ranked output), and provide feed- 
back to the user to encourage search strategy 
modification. 

Frye, Littman, and Soloway, (1988) discuss the 
interface issue from the perspective of intelli- 
gent tutoring systems, pointing out that the 
interface is more important in educational sit- 
uations than elsewhere for several reasons: 

. . .the interface niust provide an oitry to the 
conte}it dotnain rather than vice versa. In other 
types of software, the user typically knou)S what 
applicatioft a prograrn is meant to have and can 
use that knaivledge to decipher the interface . . . 
users of ediicatioftal softxvare will not have a 
similar advantage. They zvill not have a good 
understanding of the domain being taught and 
so will not have that entry to the interface ... A 
further requiranent for an interface to a piece of 
ediicatioml softzvare is that it mi4St be smsitive 
to the studaU's general kijou^ledge and/or devel- 
opmental level Givai that the interface must 
introduce the user to the domain, and not the 
reverse, the interface zvill need to provide for 
variatiofis in the skills of different users, (pp. 
452-3) 

Of interface studies, Frye et al. (1988) is one of 
the few that considered the educational envi- 
ronment as having distinct characteristics. 
These authors conclude that an evaluation of 
students' cognitive processes should be the 
basis for the user interface. Recognizing that 
this may often seem difficult in a practical 
sense, the argument for continuing to attempt 
to develop such an evaluation or model is 
simply that it will lead to more usable and 
effective computer-based learning environ- 
me/tts. 



Hypermedia interface issues 

The interface issue cannot be complete with- 
out a consideration of hypermedia. When 
articles are being written with titles such as 
"Lost in Hyperspace: Cognitive mapping and 
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navigation in a hypertext environment" 
(Edwards & Hardman, 1989) it is obvious that 
the cognitive approach to interface design is 
also being applied to the structurally complex 
'hyper' environments. 

Maule (1990) in a short article entitled "A 
review of current research in hypermedia 
information design/' describes the human- 
computer interface to the kypermedia 
environment as being "a kind of discourse for 
which no agreed- upon model exists . . . futur- 
ists project a time of 'intelligent' graphical 
interfaces, capable of maintaining and apply- 
ing information about users to increase the 
effectiveness of the systems" (p. 7). 

In The Interactive Learning Revolution (Barker & 
Tucker, 1990), the perspective is taken that 
"the user is back in control because the com- 
puter has been tamed . . . digital technologies . 
. . will unlock the knowledge bases of the 
world" (p. 15), and describe how "educators 
and trainers have leaped to the conclusion 
that interactive multimedia was invented sole- 
ly for them. Not so! . . . [it is] . . . the future 
interface of the next generation of personal 
computers" (p. 21). 

Jonassen and Wang (in Branyon-Broadbent & 
Wood, 1990) define hypermedia as 

.asoft techjwlogy for organizing and storing 
information in a knoioledge base to be accessed 
and generated nonsequentially by authors and 
users . . . although hypermedia did not evolve as 
an instructional medium, its characteristics 
mimic the associative properties of the mind, 
making it a potentially paioerful new learning 
technology, (p. 156) 

Although generally enthusiastic about the 
ability of hypermedia to provide an environ- 
ment for learners that is conducive to many 
types of learning (including information 
retrieval, knowledge building and restructur- 
ing, and problem solving), these authors 
clearly acknowledge the potential of the 
hypermedia environment to create difficulties 
in navigation, and cognitive overload 
for the user. 

The number of learning options available places 
on learners increased cogrntive deinands that 



they are often unable to fulfil Hypermedia 
browsers jnust be able to monitor their own cmn- 
prehension of the information presented in the 
hypermedia, select appropriate strategies for cor- 
recting any misconceptions, and develop 
information-seeking strategies that facilitate 
integrating infonnation arid synthesizing infor- 
mtionfrom the hypermedia. These are knoxon 
as meta-cognitive strategies, and they require 
additional effort on the part of the browser. We 
know that good learners tise them and that poor 
learners do not. Browsing hypermedia places 
signifi'^ant danands upon the user, demands 
that may take mergyfrom the more important 
process of learning, (pp. 161-2) 

Jonassen and Wang (1990) also discuss the 
issue of learner control (which was similarly 
addressed by Relan earlier in this section) and 
state that "the research on learner control has 
not generally supported any learning benefits. 
This is especially true with average and 
below-average learners" (p. 162). Here, it 
seems, is a suggestion that cognitive and 
meta-cognitive strategies (acquired almost 
magically by "good learners") cannot be 
encouraged in poor learners, and that the only 
answer is to facilitate learning by giving con- 
trol either to a teacher or to a structured 
system. In fact, we are back to the earlier 
debates raised in the section of the paper enti- 
tled: What does cognitive science tell us about 
learning? What is the status of strategy use, 
especially regarding generalization and trans- 
fer? Clearly, learners lacking a system of 
strategy use will become lost if given control 
in an unstructured environment (as hyperme- 
dia often is), but the question (or perhaps the 
challenge) remains: do we measure success by 
the amount of content acquired? Or by the 
learning strategies developed? 

In summary, research indicates that an appro- 
priate interface seems to have the potential to 
facilitate strat ^ use. There are many reasons 
why interfaces should therefore be designed 
to do so to the greatest extent possible. For 
hypermedia systems in particular, because 
they are designed to link ideas in a way that 
will not be familiar to many learners, it 
becomes even more important to ensure that 
the learner has some way of synthesizing, 
interpreting and using the information avail- 
able. Perhaps it is appropriate at this point to 
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re-acknowledge and emphasize the ideas 
expressed by Brown (1989), and Brown, 
Collins, and Duguid (1988, 1989), described in 
the section of this paper entitled: What does 
cognitive science tell us about school? How 
can we best provide an authentic learning 
experience for children? 

The next major section of this paper (Impact 
of technology, information systems and 
telecommunications) will describe how tech- 
nology (particularly information technology) 
is likely to be part of the real-world environ- 
ment for most childreri~if not right now then 
very soon. It is even more likely to be preva- 
lent by the time they reach the workplace. 
The argument that technology is part of an 
authentic learning environment is quite con- 
vincing. 

From the literature reviewed on interface 
issues, there seems to be a common message 
relating to users of a technological (or any 
other) learning environment. This message is 
derived from a cognitive perspective, and can 
be simply stated: design the interface based 
on an understanding of the user's mental 
model(s). Note that this does not confine the 
designer to replicating (through technology) 
functions that the user is already familiar 
with. It does not make sense to evaluate the 
feasibility of a prop>osed bridge based on an 
observation of the number of people who 
swim across the river at that particular point 
(Lewis, 1990). What designers must be aware 
of is how (or whether) users will be able to 
incorporate the features of a new system into 
their existing model of the world. 

It is almost surprising to find a consistent 
message across research that is derived from 
such diverse technology users as highly moti- 
vated library technologists, office workers, 
distance education students and very young 
learners. In summary, the cognitive perspec- 
tive of human-computer interface seems to be 
fairly robust. It is less clear whether there is 
(or will be) any definitive answers regarding 
the "Ideal Interface," how to identify user 
preferences, or how to determine the opti- 
mum degree of learner control. Much of the 
literature suggests that what a user requires 
from a system changes as he or she gains 
expertise, and that a single user might apply 



different strategies for interacting with a sys- 
tem in different situations. There is probably 
a message here about the need for flexible sys- 
tems iind further study of the effectiveness of 
such systems. 



Summaiy of the cognitive perspective 

To summarize, the cognitive perspective 
always makes reference to the individual and 
to his or her interaction with information. An 
understanding of the individual, the informa- 
tion, and the context in which that 
information will be acquired and used can 
facilitate the design of learning activities and 
learning environments, including environ- 
ments which require a human-computer 
interface. 
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Impact of Technology, 
Information Systems 
and Telecommunications 



There is an abundance of literature, both pop- 
ular and scholarly, that addresses the ways in 
which the increased access to information will 
impact our society. For example, Tiffin (1990) 
states that "Over the next decade, telecommu- 
nications will affect every aspect of developed 
societies including education"/' and More 
(1990) describes the changes to our economy 
and the new thinking tools that will be 
required: 

Man}/ govemmmis have begun to recognize - 
some would argie, rather belatedly - that a 
nation's very ecojiornic performance zvill be criti- 
cally affected by success in mastering, expJoiti7tg 
and marketing new developments in information 
technology and systems. The critical factor, 
however, is not technology or ecowmics alone, 
but rather the human factor ijtvolved . . . infor- 
mation systems are both culturally determined 
and culture-determining, {pp. 311-2} 

Perhaps the attitudes, skills and knowledge 
needed for participation in the information 
age can be acquired by students through a tra- 
ditional curriculum. Perhaps it is the 
responsibility of industry to equip workers to 
meet the specialized requirements of the 
workplace. If these statements are true, there 
might still be reasons for using information 
systems and computer mediated communica- 
tions in the classroom, but in this section of 
the paper, literature will be presented that 
indicates that the traditional curriculum does 
not adequately prepare students to take their 
place as functioning ap.ults in an information 
society, nor does it provide them with the 
ability to make use of specialized training 
environments that they will encounter when 
they reach the workplace. 

For instance, Berryman (1990) states that in 



the fast-changing labour market (where job 
requirements are changing more rapidly than 
ever because of advances in technology) there 
is an increased need for learners to develop 
lifelong learning skills. "Employer sponsored 
training reinforces rather than reduces the dif- 
ferences in educational attainment among 
employees. Well educated people are not 
only the most likely to find employment, but 
also the most likely to receive training from 
their employers" (p. 38). This article also 
stresses some of the same ideas expressed in 
other parts of this literature review, including 
the need for learning to take place in an 
authentic context, the need for learners to 
work collaboratively and with the seme kinds 
of assistance they will have in an authentic 
environment, and the need for learners to 
become effective problem-solvers and strategy 
users, 

McClintock (1988) states that because of com- 
puter technology we are undergoing a 
"transformation of a culture of memory to one 
of intelligence . . . people have begun to learn 
how to process intelligently the information 
they neod through objects external to living 
minds" o. 351). 

A perspective similar to both Berryman and 
McClintock is taken by Dede (1987, 1989) who 
states that 

. . . human stretigths in partnerships betiveen 
people and cognition enhancers involve skills 
such as creativity, flexibility, decisim making 
giveft incmnplete data, complex pattern recogni- 
tion, infcnnation evaluation/ synthesis, and 
holistic thinking. Such higher-order maital 
attributes might become a new definition of 
human intelligetice . . . the goal of teaching the 
basics would shift frotn performance fluency to 
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providing a cognitive underpinning for sophisti- 
cated probletn recognition and unusual problem 
solving , . . 'learning while doing' would become 
a more significant component of education 
because cofnbined computer aiid telecommunica- 
tions techfiologies allow delivery of instructional 
services in a decentralized manner, (p, 23) 

Very recent articles in Canadian Business 
Magazine, such as Allan (1991), Ziedenberg 
(1991), and Litchfield (1991) also describe a 
technologically sophisticated workplace that 
now demands and will continue to demand 
workers with greater higher-order thinking 
and literacy skills that the education system is 
currently able to produce. Clearly, there is a 
message from many sources that technology is 
having an impact that reaches into the class- 
rcx)m. 

Emihovich (1990) describes the metacognitive 
skills provided by a computer environment as 
'cultural capital/ and states that "those who 
lack this resource will fall further and further 
behind" (p. 232). But how should the comput- 
er environment be stmctured? How will we 
ensure equity of access to these important 
skills across gender, socio-economic or other 
types of barriers? 



Why get kids involved in telecommunica- 
tions or information technology? 

It would be very effective at this fX)int in the 
literature review to cite a few definitive stud- 
ies that indicate how computer-mediated 
communications or information technology 
can be incorporated into a variety of curricu- 
lum areas with the ultimate result of 
providing learners with better learning strate- 
gies, increased collaborative abilities, more 
higher-order thinking experiences, improved 
problem solving and so on. At present, such 
studies do not seem to exist. The audience for 
CMC and IT up to this point has typically 
been the university student in a distance edu- 
cation course, and it can be argued that results 
of existing implementations may not be gener- 
alizable to other groups. Unfortunately (or 
perhaps tantalizingly), there are more ques- 
tions and hopes than answers at this px)int in 
time. 



For instance, there is the eloquent question: 
"what can be done with computer-mediated 
communications that will be really valuable 
educationally in our complex, highly interde- 
pendent, uncertain world?" formulated by 
Boyd (1990, p. 271). Boyd's paper describes 
five generations of computer-mediated com- 
munications, and explains that most of the 
educational applications today are still at the 
level of the first generation. First generation 
CMC includes messaging, conferencing and 
information access without the evaluation, 
animation, graphics, sound and high level 
logic (e.g., intelligent keyword searching, 
automatic logical insertion of topics pointers, 
etc.) that are included as second generation 
functions. Third generation CMC includes 
first and second generation functions, but 
also incorporates structures to support coop- 
erative work systems (e.g., distributed 
editing). Fourth and fifth generation CMC 
include virtual environments and will "pro- 
vide participatory and anticipatory learning 
opportunities which will empower people to 
deal rationally, and perhaps beautifully, with 
the challenges of complexity and uncertainty" 
(p. 273). 

The next section of this literature review will 
describe some of the actual educational appli- 
cations of computer-mediated 
communications and information technology. 



A sample of educational applications of 
CMC and IT 

Recent issues of The Ccnnputing Teacher have a 
section devo*-ed to classroom experiences in 
the implementation of telecommunications 
(for example, Rogers, Andres, Jacks, & 
Clausen, 1990, Lake, 1989-90, and Kurshan, 
1990), These are typical of various other 
reports (including Thompson, 1990, 
Yurkovick, 1989, Waugh & Levin, 1989, and 
Goldberg, 1988) and ISTE, USWest and simi- 
lar telecommunications publications, which 
either describe technical network issues (fea- 
tures of various networking systems, 
hardware requirements and so on) or describe 
what was done in particular classrooms (or 
distance education environments) and offer 
guidelines (such as lesson plans) for others 
who want to do similar things. Often these 
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reports include testimonials describing how 
motivating and horizon-expanding the experi- 
ence was, as well as cautions about the 
possible pitfalls and black holes that can be 
encountered. This type of literature tends to 
lack objective evaluation of I ? educational 
impact, probably because it is all so very new. 
This literature does, however, quite often 
relate specifically to the public school learner 
who is the target audience for the SITP. 

Another type of literature discusses educa- 
tional (and other) paradigm shifts brought 
about through the use of conferencing sys- 
tems and information technologies (for 
instance, Harasim, 1990, Mason, 1988, 1989, 
and Ronoiszkowski, 1990); in a telecommuni- 
cations environment, the content or quality of 
information (such as a message in a confer- 
ence) becomes more important than the status 
or personal characteristics of the source; new 
information technologies include cooperative 
knowledge bases which allow users to add 
and access information at any level of detail 
desired. Through telecommunications, the 
roles of instructors and students are much less 
distinct than in traditional classrooms. This 
literature is based on thoughtful observations 
of the apparent effect of the new technologies, 
but even here, the impact on learning is not 
usually objectively explored. This literature 
tends to deal with college or graduate level 
students. For instance, although 
Romiszkowski mentions that the implementa- 
tion of such technology should reach down to 
at least high school, he feels it is less likely to 
have impact at the elementary level, mainly 
for social reasons. Teles (1991), however, has 
successfully used audio conferencing with 
children as young as Grade 6, and describes 
its value as an interactive learning tool. 

Perhaps (as has been the case with the early 
phases of many innovations), the somewhat 
subjective initial reactions to computer-medi- 
ated conununications or information 
technology-oriented learning environments 
will form the basis for a more in-depth analy- 
sis of the positive and negative impact on 
learning and will eventually lead to tlie devel- 
opment of increasingly effective CMC and IT 
activities. 

A third type of literature describes some of 



the frameworks that have been developed to 
guide the inclusion of CMC and IT in public 
school curriculums. It is perhaps worth not- 
ing that in these examples, the technology is 
specifically related to cross-curricular goals 
for learning and that there are clear implica- 
tions regarding the evaluation of 
effectiveness. 

Van Weering and Plomp (1991) describe how 
information and computer literacy (ICL) is 
now (based on the recommendations of a 
committee formed ii\ 1987) being included in 
the curriculum for Grades / to 9 students in 
the Netherlands. The stated objectives, relat- 
ing to information technology (IT) and 
computer use, are as follow: 

(1) students develop insight into processes of 
purposeful data collection, processing and 
retrieval, and are able to deal zoith data and 
informatimi in a way that detnonstrates 
understanding; 

(2) students have correct functional pictures of 
data processing systems (such as the comput- 
er) and, on the basis of these, can use the 
systems; 

(3) students know hozv and lohere IT is being 
applied, and are able to use various applica" 
ticms of IT; 

4) students have insight into the social impact of 
IT. (p. 18) 

"Problem solving is still one of the most 
important feature of ICL, however, no longer 
in the sense of solving a problem with respect 
to programming, but instead in being able to 
find information by using appropriate new 
information technologies" (p. 21). The above 
goals are to be met through integration with 
the curriculum, and although preparatory 
skills can be provided through a separate 
course, the majority of ICL experience is 
gained through participation in other course- 
work such as math, science, language arts, 
social studies, technology and arts. 

J.S.A. Anderson (1991) describes the cross-cur- 
ricular IT competen / that Northern Ireland 
introduced in 1990 as a statutory element in 
compulsory schooling for students aged 4 to 
16. The five objectives are incorporated into 
the following statement: 
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Pupils should be able to }mke appropriate and 
effective use of IT to: 

- develop, modify and co^nmunicate ideas; 

- create, collect, store, retrieve, validate, change, 
interpret and presejit ijifomiatiofi; 

- manipulate, investigate and create models of 
situation; 

- measure and control eletnejits of the physical 
environrjwit; eimluate the impact of IT on 
themselves, other individuals, organizations 
and society. 

These objectives are to he wholly or mainly deliv- 
ered through the teaching of the contributory 
subjects of the curriculum; their attainment by 
pupils is to be assessed in the ccmtext of the use 
of IT us a tool in the contributory subject, (p. 23) 

Students are assessed for competency with 
reference to 10 levels of attainment. 

North (1991) describes the challenges faced by 
the staff of schools in Northern Ireland relat- 
ing to the development of effective 
implementation strategies including staff 
development and the accommodation of 
cross -curricular linkages. This project is obvi- 
ously in an early stage, and one anticipated 
outcome is that the integration of IT will be a 
catalyst for fundamental changes to the cur- 
riculum. 

These varying types of literature (and there 
are probably many other 'types' that have not 
been described) suggest that many educators 
are enthused about using CMC and IT in the 
educational environment in different ways 
and for different reasons. Almost everyone 
involved has experienced frustration of some 
sort. Evaluation of such projects ranges from 
the extremely subjective to the relatively 
structured. Some of the rationales for CMC 
and IT educational projects have been devel- 
oped by an individual or small group of 
teachers; others have been developed through 
the consensus of a large group and are being 
implemented at a national level. 



How shall we interpret the status of educa- 
tional CMC and IT? 

The types of implementation projects 
described in the previous section seem to 
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point out that CMC and IT have characteris- 
tics that have convinced a number of 
educators to view them as being very impor- 
tant technologies for educational purposes. 
Some educators are scrambling to tame these 
technologies so that they can use them effec- 
tively. Since such projects are still in the early 
stages, problems and unresolved issues are to 
be expected and many new questions are like- 
ly to arise. 



The current status 

Scardamalia and Bereiter (1989) discuss the 
growing body of projects in which students 
link electronically to other schools or coun- 
tries, to large databases or to experts. These 
authors describe two tyjDcs of problems: 

I . the filter problem . . . xoays must be found to 
reduce redundancy, to sort out questions that 
can be ansxoered locally or inexpmsively . . . and 
generally to make every demand on the netxvork 
count for as much as possible . . . land] 2. the 
inquiry upgrading problem . . . xohich states that 
learners involved in media systems must first 
nigage in serious thought and study, and not 
expect the 'experts* to do their xvorkfor them. 
{]yp. U-15) 

In a discussion of computer-mediated com- 
munication in education, Boyd (1990) states, 
"it is sometimes quite important to restrict the 
facilities available for a given course or lesson 
to the smallest subset of those available, the 
ones which are really needed, so that partici- 
pants will not be distracted by 'noise' or 
frequent calls upon them to make really 
unnecessary decisions" (p 271). 

The comments above suggest that CMC and 
IT activities should be purposeful and clearly 
related to the kinds of learning that have been 
described in the 'cognition' section of this 
paper. Not all questions require the answer of 
a distant expert. In some cases, collaboration 
might be just as effective across the hall as 
across the ocean. Learning requires the active 
construction of knowledge~not just the receipt 
of a canned answer from an expert. A confus- 
ing computer system can detract from the 
educational purpose of the activity. 
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The future 

Perhaps the first type of place to enquire 
about the future of computer mediated com- 
munications and information technologies 
should be a 'think tank' such as Xerox's Palo 
Alto Research Center (PARC). "PARC scien- 
tists believe that information retrieval and use 
will be lo the 1990s what text editing was to 
the 1980s. Information retrieval is not an end 
in itself, but part of some larger activity" 
(Clarkson, 1991, p. 278). What are the 'larger 
activities' in education that can make use of 
the benefits of CMC and IT? Tlie fuhjre almost 
certainly includes continued research and 
work to come to a better understanding of 
how CMC and IT have an impact on learning. 
As some questions are answered, more will be 
identified. This section raises possible ques- 
tions for current research projects; in 
particular it addresses questions that are not 
mentioned in other parts of the paper. 

Nipper (1989) claims that "distance education 
has always advantaged the highly motivated, 
and often educationally privileged adult 
learner. In important resf>ects, this feature is 
exaggerated even further by CMC" (p. 72). 
Does this indicate that such technology is 
inappropriate for the less privileged younger 
learner? Are there ways in which CMC activi- 
ties can be designed to accommodate less 
motivated learners? 

Muckler (1987) suggests that literature 
retrieval "systems claim to heighten produc- 
tivity without a clear definition or supporting 
data on what is meant by productivity" (p. 9) 
and forms some "questions for future sys- 
tems: Does the new system help get the work 
done? Is the system really productive? To 
know this we must carefully define productiv- 
ity and measure it. And be prepared for 
disappointment. Should we allow enthusiasm 
to substitute for performance?" (p.ll). 
Muckler's paper does not limit its focus to the 
educational use of CMC, but it seems that the 
notion of the 'work' of learning, the necessity 
of defining 'productivity,' and the caution 
against unsubstantiated enthusiasm are all 
concepts that can be appropriately applied to 
education. How can we measure educational 
productivity? Is enthusiasm always a positive 
factor in education? 



If, as Somekh (1989) describes, "one of tlie 
main problems [of CMC] lies in getting people 
to use it" (p. 242), it may be a problem of rais- 
ing their awareness or it may be that the 
educational advantages do not yet outweigh 
the disadvantages. Chesebro & Bonsall (1989) 
offer a critique of the information society, 
including the statement that "massive quanti- 
ties of data are slowly being defined as 
information and ultimately as knowledge, and 
the consumption of mass quantities of infor- 
mation is being equated with understanding" 
(p. 235). A similar critique is offered by Sless 
(1985): "it is readily obvious that more effi- 
cient forms of information transmission are 
not necessarily going to lead to better commu- 
nication, but to more, though undoubtedly 
faster, misunderstandings" (p. 112). These 
criticisnns raise some very important questions 
for educational research. How can educators 
ensure that CMC and IT will not become a 
source of frustration or misunderstandings for 
learners? 



Summary of the impact of technology 

This section has attempted to provide an 
overview of the issues related to the use of 
technology in education. In particular, effort 
was made to identify some of the questions 
and concerns generated by the current appli- 
cations of CMC and IT. In a very real sense, 
the investigation is just beginning. Pea and 
Soloway (1987) compare 'folk theories' of 
medicine in the early 1800's to current 'folk 
practices' in education and clearly point out 
the need for educators to acquire and use a 
more reliable understanding of learning. 

Whot transfonr^ medical practice xvas rnedical 
sciatce, requiring the best minds in highly- 
focused anpirical attacks on the mture of disease 
and biological systems, and the use of the most 
advanced techmlogies available, tuned specifi- 
cally to the problems' dimetisions. 
Understanding the xvorkings of?wnd in learning 
in society is 0)\e of the major frontiers of science. 
Until the co?uplexities of education are better 
appreciated, we cawiot expect popular aithusi- 
asm about research arid develop7nefit for 
educatioml technologies, (p. 35) 

Perhaps research in educational technologies 
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is not popular, but it can be argued that the 
education potential of CMC and IT cannot be 
put to best use until their impact on learning 
becomes better understood. 
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Educational Technology 
Research and Evaluation: 
Issues and Methods 



For this paper, an educational theoiy will bo 
defined as a system for understanding how 
some aspect of learning takes place in some 
type of educational environment. Theories 
can be supported by the results of research, or 
they can be disconfirmed or modified through 
research findings. Unlike the 'folk theories' 
described by Pea and Soloway in the previous 
section, valid educational theories must be 
more than 'patchwork solutions,' or collec- 
tions of unrelated observations-they must be 
consistent, coherent and explicit. Valid theo- 
ries have explanatory power within an 
identified domain. 

A valid theory (which would provide a much- 
needed framework) for educational 
technology research would have to incorpo- 
rate a structure for the formulation of 
questions about how learning occurs (what 
are the cognitive and motivational aspects?), 
questions about the potential roles and func- 
tions of technology (is technology a tool? 
medium for expression? how does it shape 
us? where is the locus of control?), and ques- 
tions about the intended 'outcomes' of the 
interaction of the learner with the technology. 
Are we interested in looking at the impact of 
technology to support narrowly defined learn- 
ing outcomes, such as improved test 
achievement, or should we be concerned with 
more broadly defined outcomes, such as 
improved problem solving abilities? Research 
questions that are clearly framed by a theoret- 
ical perspective will provide results that are 
more easily integrated and applied. 

All educators operate on the basis of a theoret- 
ical perspective, although often they do not 
explicitly or completely identify the compo- 
nents of the theoretical framework involved. 
The educator who holds an explicit theory is 



open to the challenge of having to reformulate 
that theory when presented with observations 
or data that do not confirm it. Holding an 
implicit theory can present problems because 
it is relatively easy to ignore evidence which 
contradicts the theory. 

Phase i of SITP has been bastxi on a theoretical 
framework that is largely implicit. Tliere is an 
assumption that technology can be used to 
effectively support education, but the con- 
cepts described above (learning, functions of 
technology and intended outcomes) have not 
been fully identified. Tins leads to several 
research issues that must be addressed. The 
most important of these is the identification of 
educational expectations against v*^hich the 
effectiveness of technological innovations can 
be viewed. Other important issues for the 
project include establishing appropriate focus 
areas for research and selecting methodolo- 
gies that are suited to those topics. Some 
methodologies involve setting up control and 
treatment groups for comparative purposes, 
others (involving less intervention) depend on 
ethnographic observational techniques. The 
research methodology selected must provide 
reliable and objective information, as well as 
maintain credibility with the audience of edu- 
cators who might ultimately be involved in 
implementation related to the research results. 

This section of the paper will outline some of 
the educational research issues that could 
influence the current project. A few general 
issues and some issues specifically related to 
technology will be discussed. Methodologies 
used in research studies that could be related 
to SITP will be described. 
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Issues in educational research 

Cuban (1984) articulated the sentiments and 
frustrations of nr^any educators and education- 
al researchers. Over the years, hundreds of 
research studies have been conducted, yet 
very few have produced any significant 
changes in practice. Some fairly recent trends 
that may help to promote greater overlap 
between research and practice will be 
described in this section of the paper. 

Validity is an important issue that is closely 
related to the methodologies of educational 
research. Classroom teachers might be more 
willing to implement research findings if they 
believed that research methodologies provid- 
ed results applicable to their own situation. A 
current trend in the design of educational 
research involves greater teacher participation 
which seems likely to lead to greater accep- 
tance of research findings. 



Naturalistic versus controlled research: can 
they co-exist? 

Frye, Littman, and Soloway (1988) describe 
the fact that research methods need to evolve 
in congruence with cognitive theories: "even 
though educational evaluation is an estab- 
lished field, methodologies have not been 
developed for evaluating educational systems 
that attempt to teach students to "under- 
stand"-rather than simply to get the right 
answer" (p. 451). 

Methods are being developed that may, in 
fact, indicate that such change is taking place: 
"cognitive strategy research is moving away 
from the lab and into more naturalistic educa- 
tional settings" (McCormick, Miller, & 
Pressley, 1989, p. v.). 

Symons, Snyder, Cariglia-Bull, and Pressley 
(1989) describe recent strategy research as 
being 

. . . extremely infonmtive about intervmtioiis 
that have real-world utility. One reasofi is that 
contemporary strategy researchers and theorists 
recognize hozv various research cjuestioris about 
strategy instruction are related to one another. 
A second reasoti is that researchers are gaining 



savvy in designing studies that answer these 
questions, (p. 17) 

A straightforward research methodology is 
described: a 'treatment' group participates in 
particular learning activities, while a control 
group does not. Outcomes are compared. 
Although the "complex reciprocal interactions 
between teachers and students that are not 
always easily characterized or easily con- 
trolled for research purposes" (p. 18) can 
sometimes be an obstacle to getting clear 
answers to research questions, there is general 
agreement that controlled research methods 
applied in the classroom can yield worthwhile 
rcoults, particularly when part of a long-term, 
multi-experiment endeavour. 

These authors describe how this can take 
place: 

What are good classroom studies like? They are 
motivated by clearly artiadated theories and 
i^nportant instruction issues. Actual teachers 
carry out the insirucliGn, nnth teaching occur- 
ring over an extended period of time. The 
studies are true experiments (i.e. there is random 
assigtiment of participants to instructed and 
cmitrol cmidittons), A variety of dependetit mea- 
sures are collected in order to assess the 
multidiynensional impacts of the intervaiiions. 
(p. 22) 

Laboratory-based learning research is also 
moving toward more relevant tasks, and 
when combined with the results of classroom 
research, there may be real reason for enthusi- 
asm. 



Research issues in educational technology 

The literature suggests that a comprehensive 
theory of technology in education has not yet 
been developed. Describing information tech- 
nologies in education, the 1989 report of the 
OECD concludes "There has been little 
research into how students learn with com- 
puters, what they learn when they interact 
with computer-based materials or how their 
learning could be improved by the use of the 
information technologies" (p. 15). 

Acker (1989) believes that due to the problems 
in keeping up with technological changes. 
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implementation of technology must now 
come first and "theory construction is left for 
whomever has the time later" (p. 319). The 
contrasting point of view is presented by 
Salomon (1989) whose concern is expressed in 
the following statement: "Are we back to tlie 
atheoretical empiricism which has character- 
ized television research a few decades ago?" 
(p. 250), 

Chaiklin, Hedegaard, Navarro, and Pedraza 
(1990) describe a theoiy-based approach to 
computer use in education that includes the 
following principles: 

I. All educatimal activities should be organized 
toioard helping stiidmts develop a theoretical 
approach to subject matter 

1, The inotivatio7i for investigating a particular 
topic in the curriciihan should come from the 
subject matter and the zvay it has be developed 
with the students 

3. Ccnnputer use should be embedded as a part of 
a larger educational program, (pp. 270-1) 

The fonnulation and application of [educational] 
goals are aided sigixificantly by having a theoret- 
ical perspective about pedagogical practices, 
(p, 273) 

Although, in a fast-changing environment, 
theories may never even come close to being 
fully developed before implementation must 
take place, it seems unproductive to complete- 
ly abandon the attempt to build on a 
theoretical framework. Tlie cognitive theories 
that have been described in this paper, moti- 
vational theories, theories of cost 
effectiveness, theories of change and others 
might all be appropriate starting points for 
evaluating the impact of technology. 



Frameworks for educational technology 
research 

Research in the use of technology for educa- 
tional purposes has not always been 
structured on the foundation of theoretical 
frameworks. Some of the frameworks used 
have been less coherent and comprehensive 
than would be the ideal. The following sec- 
tions describe examples of some frameworks 
that have been developed in educational tech- 



nology. It is possible that some of the ideas 
from these frameworks could help with the 
identification of research questions, the for- 
mulation of research methodologies, and the 
establishment of a more complete theoretical 
perspective for SITP. 

Frameworks such as those described in the 
next section provide categories which help to 
organize the relevant issues, althougn they do 
not really achieve the objectives of a valid the- 
ory for educational technology. More 
centralized frameworks, such the ones pre- 
sented in the following section based on the 
dimension of cost-effectiveness, can be related 
to all other dimensions of a research study. 

Organizing educational technology research 

Pea & Soloway (1987) have addressed the fun- 
damental roles for education science and 
technology. "Research in the cognitive, social, 
instructional, and computation sciences has 
brought about a fundamental change in our 
thinking about learning and teaching" (p. 30). 
These authors provide a 

. . . framczvork for organizing the R&D direc- 
tions considered most promising by experts in 
the field: 

1 . changing tvhat students do; 

2. changing hozo we track leamiyig; 

3. changing tvhot's taught; 

4. changing zvhat teachers do; 

5. changing the school mvironmmt. (pp. 31-32) 

A different type of research framework is pre- 
sented by Disessa (1987) to assist in 
understanding the impact of computers on 
education. She argues that a real revolution 
far beyond just "getting computers into 
schools" is underway which emphasizes the 
importance of the following issues in planning 
and research: 

- acceptance of complexity (no simple mecha- 
nisms of explanation) 

- shift in phenomenological focus (to domain 
specific) 

- shift in methodology (new major focus is 
protocol analysis - moment by moment flow 
of problem solving activity). 
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"We must understand education as a complex 
interacting whole that must reach a new equi- 
librium in response to the niany major 
perturbations that will be supplied by infor- 
mation technology" (p. 355). 

Emihovich (1990) suggests that the following 
questions are important in a study of comput- 
ers in education: 

Who benefits from computer usage? and its 
corollary questioris, Are these benefits equally 
distributed? V/hat curricular and staff develop- 
ynent changes are required? What changes in 
teacher-student relationships can be expected? 
How much control (if any) do teachers have over 
implementation? and most importantly, VJhat 
aspects of children's learning do we really desire 
and expect computers to change? (p. 226) 

She discusses some Logo studies that did not 
show positive effects, and concludes that per- 
haps they asked the wrong questions. The 
studies that are viewed as more useful had the 
following characteristics: 

a. the researchers assessed educational outcomes 
aver time, instead ofexamitting short term 
gains 

b. the researchers developed case studies ofindi- 
vidtwVs thinking processes 

c. the researchers relied on teacher judginaits as 
an important source of evidmce, combined 
xvith information from other sources, 

. . . a7iy research study, experimental or other- 
xvise, begins with subjective judgments about the 
importance (or worth) of the questions being 
asked, the ways in which evidence is collected, 
the meaning of the tools or instruments being 
used, and the choice of analytical strategies, 
(p, 233) 

Cost effectiveness and educational 
technology 

Another important consideration in a realistic 
framework for educational technology (and 
related research) is the issue of cost effective- 
ness. Clearly this issue should not be 
separated from other considerations. It may 
be quite appropriate to try out new and 
expensive technologies in a research environ- 
ment as long as the costs and benefits are 
included in the final report which u,a then 



serve as a guideline to future implementation 
projects. It is somewhat surprising to find 
that although de^. oion-makers urgently need 
information about the cost-effectiveness of 
new technologies, there is little literature 
available on this topic. 

Moonen (1990) prepared a report for ETC 
which summarizes some of the literature on 
the cost-effectiveness of educational technolo- 
gy. He states "available literature deals 
mostly with the cost aspect, although the 
effectiveness aspect is very often mentioned in 
the title of articles" (p. 3). Since educational 
expectations or goals are often not very explic- 
it, it becomes difficult to know whether or not 
an implementation of technology meets those 
goals in a cost-effective manner. It is not even 
easy to really measure the true cost of tlie 
implementation. There are usually hidden 
costs, and prices can be very unstable. 

Moonen formulates a ratio using specific mea- 
sures of effectiveness (relating to educational 
goals achieved) including: 

- number of studatts who complete a program 

- nimber of potential dropouts who graduate 

- number of graduates placed in appropriate jobs 

- test scores (using appropriate instruments) 

- studait assessments of a program. 

These measures can be related to the impact of 
a new technology whose cost is known. They 
are quantifiable measures of educational 
goals. Moonen's conclusion is that "We 
should start Sf>ecifying what we want in terms 
of costs and effectiveness measurements. The 
model developed above is a first step to sup- 
port our expectations. From that perspective 
many research projects can be defined" (p. 
30). Technological, or any other type of 'solu- 
tions' need to be compared to alternatives that 
might achieve the same goals for less cost, but 
this cannot happen if the goals have not been 
identified first. 

Van Horn (1991) describes the 'student pro- 
ductivity argument' as follows: information 
technology must be used directly with the 
learners and not directed to the teachers. "We 
must concentrate on the 25 instead of the one. 
When we invest in new technology, we must 
invest in new learning systems, not in new 
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teaching systems" (p. 528). This is based on 
the premise that although students do not 
receive a salary, and have therefore often been 
omitted as a cost element in the evaluation of 
school productivity, they represent a real cost 
to society that will increase if their learning 
environment becomes less effective in meeting 
their needs. Here, as in Moonen's framework, 
productivity cannot be evaluated outside of 
the context of well-defined educational expec- 
tations and goals. 

The computer-mediated communications and 
information technology environments are nei- 
ther cheap nor easy to implement. Berk (1990) 
describes the current situation: 

. . . although there would seem to be some inter- 
esting possibilities for pedagogical developments 
that could take place in network envircnvnaits, 
and could not take place in other eiivirownents, 
these possibilities have scarcely been touched. So 
networks become an expensive addition, not con- 
tributing much to the learning situation. But, 
like many of the early stages in the history, they 
reflect a potmtial that might be realized if the 
final stage of history is ever attained, (p. 103) 



Research and evaluation methods that might 
apply to SITP 

The following sections give a sample of stud- 
ies that relate to technological applications 
from non-educational as well as educational 
environments. They have been selected 
because they incorporate research methods 
that might be useful to SITP. 

Methodologies used outside of the school- 
based educational environment 
Diverse experimental and observational meth- 
ods have been used to evaluate issues related 
to computer use in the workplace. The goals 
of such research may not exactly match those 
of education, but some of the methodology 
might be suitable (with modifications) for 
finding out more about educational applica- 
tions and some of the results might well be 
generalizable to the educational context. This 
section of the paper touches on a few studies 
(mostly related to the improvement of human- 
computer interfaces or to the issue of 
improving human performance through com- 



puter use) that seem to be of interest. 

Cook and Ridley (1990) looked at librarians 
using the CoSy computer conferencing system 
at McMaster University. Their nnain tools of 
investigation were a survey and an analysis of 
conference use. They state that "in communi- 
cations research, it is often difficult to isolate 
the effects of the system, the environment, the 
participants and other communications meth- 
ods available. Much CMCS [computer 
mediated communications systems] research, 
as with this study, is done many months after 
the introduction of a new system" (p. 413). A 
conclusion of this study is that successful 
CMC requires task orientation, support from 
administrators, and easy access. 

As with many similar studies, Cook and 
Ridley observed and analyzed patterns of sys- 
tem usage. User difficulties were noted and 
steps to overcome them (such as ensuring 
greater access to computers) were suggested. 
Beneficial results are assumed to be an out- 
come accompanying increased usage of the 
system, but such results are not described nor 
measured or observed in the study. A section 
of the paper, entitled 'usefulness of CMCS' 
describes the ten survey categories of useful 
aspects of the system, and ranks them based 
on survey results. Respondents rated the sys- 
tem as useful for exchanging information and 
asking questions, but does not indicate 
whether this is an improvement over the pre- 
vious methods used for similar purposes. In 
fact, the conclusion is that the librarians joined 
out of curiousity, but their "continued use of 
the system would depend on direct applica- 
tion to their jobs or tasks" (p. 415). 

Cherry (1990) discusses the strengths and 
weaknesses of methods such as surveys, lab 
studies and field studies to gain an under- 
standing of database users. Surveys are 
economical and can sometimes be quite accu- 
rate but they usually lack depth and "provide 
only secondary data - what users say they do. 
This is based on their recollection of what they 
did, which is often inaccurate or not suffi- 
ciently detailed" (p. 20). In lab studies, the 
investigator has control and can be "confident 
that the observed effect ... is actually due to 
the treatment, however, the artificial nature of 
laboratory settings threatens the external 
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validity of these studies" (pp. 20-1). Field 
studies involve observation of real users in 
real environments. Tliey are more expensive 
and are less easily controlled by the investiga- 
tor, but provide ecologically valid results. 

Software monitoring is a typ>e of field study, 
but unless it is carefully planned and ana- 
lyzed, it can provide a large volume of data 
that does not answer any meaningful ques- 
tions. The conclusion is that methods are 
complementary, and should be used to sup- 
port each other. 

Another description of methodologies (which 
could be used in field studies in the educa- 
tional environments) is presented by Graesser, 
Gordon, Forsythe, and Greer (1991). This 
paper describes the collection of information 
about the effectiveness of computer interfaces 
through verbal protocols, including on-line 
think aloud, retrospective memory, and ques- 
tion-probe methodp. Verbal protocols involve 
a person describing their actions (or answer- 
ing questions) as they perform a task or 
immediately afterwards. 

One of the imjor virtues of verbal protocols is 
that they permit researchers to discover xvhat 
cogiiitive structures and strategies hwnans use 
in tasks when the researchers have no theories to 
guide their expectations; that is, verbal protocol 
methods are excellent in the discovery phase of 
research, (p. 16) 

However, since the verbal protocol situation is 
not typical of normal performance (people 
don't usually talk out loud while performing 
a task), it is necessary to validate conclusions 
derived from observations made during the 
verbal protocol process; for instance, expecta- 
tions from verbal protocols can be evaluated 
by how well they predict performance in nor- 
mal situations. 

An example of a study using a think-aloud 
methodology is a library search study con- 
ducted by Ingwersen (1984). During the task 
of an information search, librarians were 
asked to describe what they were doing, and 
why. An analysis of the librarians' records 
allowed the researcher to relate various 
searching behaviours to the models of infor- 
mation held ?)y individual librarians. The 



results have been successfully used to teach 
reference librarianship. 

O'Shaughnessy, Coskuntuna, and Kantro 
(1987) devised controlled experiments to 
investigate the human decision-making pro- 
cess when using an on-line information 
retrieval system. Participants were required 
to evaluate information about products, and 
make a purchasing decision. The conclusion 
for users of on-line systems here is that the 
human information-processing capacity is 
limited, and systems must be designed to alle- 
viate these limitations by structuring access to 
information. 

Streitz (1987) has said the following about the 
state of human-computer interaction research: 

Most of the current research is concerned with 
evaluating existing systems .it is not very 
innovative. A second strategy is to be part of a 
rapid prototyping team thus allowing to be very 
close to nexv technological developments . , .but . 
. . it is necessary to cornpletnent these approaches 
by trying to progress conceptually one step 
ahead of existing technology . . . and provide 
frarneivorks for future interactive systems. The 
possible role of psychologists or cognitive 
ergonomists xvould theft be to propose require- 
metits xvhich then have to be met by technology, 
(p. 77) 

Bamber (1990) reviews Zuboff's 1988 book 
entitled In the Age of the Smart Machine: The 
Future of Work and Pozver, which takes an 
industrial anthropological approach to study- 
ing technology use in the work place. The 
focus question, does technology rob people or 
empower people? is dealt with through a vari- 
ety of research methods, including getting 
people to draw pictures of their experiences 
with new computer systems. Observation of 
meetings, small group discussions and analy- 
sis of products were other approaches used in 
this investigation. The conclusion is that the 
impact of technology could go in either direc- 
tion. People may become empt^wered or they 
may become enslaved-it is an organizational 
choice. The caution is that the more expedient 
approach (easier implementation, less radical 
reorganization of power hierarchies) is also 
the less humanistic choice. 
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Education-oriented methodologies 

This section presents examples of two general 
types of methodologies, along with examples 
of how they can be combined to form a 
hybrid, all from educational technology 
research. The first examples (Archer, 1989, 
and Reynolds & Hart, 1989) are controlled, 
experimental studies. Also taking the experi- 
mental perspective is the report for the U.S. 
Department of Education conducted by 
Becker. The hybrid (Newman, 1990) involves 
a method that controls the outcome and 
observes the processes used to achieve that 
outcome. Three other examples involve stud- 
ies or approaches that can be categorized as 
naturalistic (Neuman, 1989, Baird & Mitchell, 
1986, Lytle & Cochran-Smith, 1990). 

Archer (1989) conducted a study of the prob- 
lem-solving, idea generating, and 
decision-making abilities of MBA students in 
computer conference groups of varying sizes. 
The groups were carefully matched for previ- 
ous computer experience and all participants 
received the same training. Each group ana- 
lyzed the same business "mini-case" through 
the conference, and tlie quality of their analy- 
ses was compared. Similar studies conducted 
in face-to-face group meetings indicated that 
problems (such as unequal participation) arise 
for a group size greater than five members. 
The conferencing methodology used was seen 
to alleviate this problem, and comparable 
work was generated in groups of as many as 
13 students or as few as four. 

Reynolds and Hart (1989) designed an experi- 
ment where children's stories (composed 
using a word processor) were evaluated 
before and after various writing procedures 
were taught. Matched groups had either cog- 
nitive mapping strategies, brainstorming, or 
outlining explained to them. The effectiveness 
of the various procedures on writing and revi- 
sion was examined. 

Meta-analysis is the process of combining the 
results of a series of studies conducted in a 
particular area. Becker (1986) described the 
inadequacy of the majority of individual stud- 
ies in the area of instructional computer use: 
studies based on survey methods lacked 
validity, classroom studies or field experi- 
ments rarely used random assignment 



techniques and were usually conducted over 
very short time periods with groups too small 
to provide meaningful results. Because meta- 
analysis (as it is usually conducted) 
incorporates studies with such defects, Becker 
(1987) used a similar, but more refined, tech- 
nique called 1:>est-evidence synthesis' and 
concluded that "the existing evidence of com- 
puter effectiveness is very scanty" (p. 24). He 
proposes a large-scale, randomized and exper- 
imentally controlled series of studies to be 
conducted over a period of three years. This 
study will use field research and survey tech- 
niques and will focus on the effect of a variety 
of types of computer use in specific curricu- 
lum areas. 

Newman (1990) describes studying the orga- 
nization of computers in schools (in this case 
the environment was classrooms with LANs) 
through a research methodology called the 
"formative experiment." In the formative 
experiment, pedagogical goals are set and the 
process required to achieve those goals is 
observed. 

Instead of rigidly controlling the treatments and 
observing differaices in the ontcoine, as in a con- 
vmtional experiment, fonnative experiments 
aim at a particular outcome and observe the pro- 
cess by which the goal is achieved . . . if the 
e)iviro}ime)it, rather than the technology, is the 
unit of analysis, changes in the instructional 
interactions, changes in teacher roles, and other 
ivays that the ediicatiojial e)wironment is 
changed are observed . . . the changes in cn'garii- 
zation brought on by technological support can 
be understood also as changes in the support for 
cogjution. (p. 10). 

Neuman (1989) discusses the potential of a 
naturalistic inquiry paradigm in the develop- 
ment of computer-based instruction: 

Three areas co7ttribute to making the naturalis- 
tic, or qualitative approach the most prcnnising 
alternative for gaining the detailed, conte> '. • 
bound information required to design miximally 
effective coursexvare: the theoretical assumptions 
underlying the 7iaturalistic paradigm, the 
methodology commcmly associated xvith it, and 
the nature of the findi7tgs that result from natu- 
ralistic studies, (p. 40^ 
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The theoretical assumptions emphasize the 
importance of context in learning, and are 
based on cognitive theories. The methodolo- 
gy includes "prolonged and persistent 
observation of students and teachers using 
courseware and engaging in introductory and 
follow-up activities" (p. 44). The nature of the 
findings "could provide the foundation for a 
wide an ay of design principles upon which 
courseware developers could draw to enhance 
the quality and effectiveness of courseware 
used in schools" (p. 49). 

The above approaches, varying in their degree 
of experimental control, are alike in that the 
researcher is external to the actual learning 
environment. A different approach, usually 
incorporating naturalistic methodologies, 
involves the teacher as the individual who 
designs and conducts the research. The 
remaining examples describe projects where 
teachers take an active role in research. 



ysis of information. Perhaps the most impor- 
tant benefit is that teachers involved in 
research projects become more aware of edu- 
cational issues, and begin to think more 
critically about the learning environment. 



Summary of research and evaluation issues 

It is clear educational technology research and 
evaluation exists in many forms and employs 
many methodologies. Research questions 
should be based on a theoretical framework 
that incorporates a definition of relevant edu- 
cational expectations. Methods are being 
devised which incorporate the objectivity of 
experimental studies and the contextual valid- 
ity of field studies. Perhaps the results of 
research employing such methods will influ- 
ence educational practice in a way that has not 
occurred to this point. 



Action research, used in the Australian Project 
of Enhancing Effective Learning, or PEEL 
(Baird & Mitchell, 1986) is a naturalistic 
methodology that engages the teacher as 
research designer and principal investigator, 
within a classroom of students who are also 
actively involved in investigating their own 
learning. Research of this type is not an 'add- 
on' to a particular program, but becomes an 
integral part of the teaching and learning 
environment. PEEL involves a framework for 
teachers and learners to examine and revise 
their everyday processes. By providing a set 
of assertions about teaching and learning, and 
activities such as journal writing and concept 
mapping, this methodology encourages the 
change from rote-learning to knowledge cre- 
ation. 



The teacher as researcher has also been 
defined by Lytle and Cochran-Smith (1990) as 
the "systematic, intentional inquiry by teach- 
ers about their own school and classroom 
work" (p. 84). Through journals, essays, oral 
inquiry, and classroom studies, teachers can 
add to the knowledge base of their profession. 
Although there has been some debate about 
the status and objectivity of this type of 
research, these authors argue that the power 
of such work lies in the continuous revision of 
questions and methods and the on-going anal- 
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Conclusion 



The intention of this paper has been to pro- 
vide a background perspective, based on a 
sampling of current literature, for the research 
and evaluation component of SITP. Although 
a wide range of literature has been covered, 
there are probably some important studies 
and perhaps a few points of view that have 
not been included in this review. 

It is interesting to note that the work of 
preparing such a review incorporates many of 
the activities that will be performed by stu- 
dents as part of the project: gathering 
information from many sources (including 
database searches), evaluating that informa- 
tion for its value in addressing a particular 
problem (how to design a research agenda for 
SITP), and finally, using (in this case, summa- 
rizing and synthesizing) that information to 
create a format that could be useful to a par- 
ticular audience. 

A large scale networking project, such as 
SITP, offers students access to information 
that would not be possible otherwise. It also 
provides an opportunity to obiierve how 
learning takes place in an information-rich 
environment. The premise of this paper has 
been that in order to evaluate new educational 
directions, there must first be a consideration 
of the potential relationship between the new 
directions and the exp)ectations, goals, and 
purpose of education. I believe that this 
requires an understanding of how learning 
takes place and a clarification of what we 
expect children to learn and why we expect 
them to learn if. In British Columbia, the 
Report of the Royal Commission on Education and 
the Year 2000 paper have addressed these 
issues. The current paper provides a perspec- 
tive (derived from a literature review) about 
specific issues related to learning through 



technology. The particular issues addressed 
have been selected because they can provide a 
framework for formulating some of the many 
questions raised by the unique situation of 
SITP. 

The literature on cognitive theory provides a 
useful perspective on learning. Literature that 
considers the society of the future, 
including the impact of new technologies, 
gives us a framework for deciding what will 
be important for our children as they move on 
in life. Perhaps a goal for all educational sys- 
tems should be to develop and use theoretical 
frameworks that incorporate the 'what, why, 
and how' of learning. Such frameworks can 
assist in decision-making about all types of 
educational innovations, including new tech- 
nologies. 

The unnecessary division between theoretical 
and practical issues would be narrowed by a 
spirit of collaboration between researchers 
and practitioners; after all, both groups are 
working toward providing effective learning 
environments for students. Everyone involved 
in the processes of education (including stu- 
dents and parents) operates on the basis of 
theoretical frameworks about (or systems for 
understanding) educational issues. 
Communication is easier when the theoretical 
frameworks are made explicit. 

This paper has included examples of various 
research methodologies to provide research 
design ideas for projects that investigate the 
impact of information technologies (such as 
telecommunications) on learning. As these 
technologies become more widely used in 
educational environments, it is important to 
evaluate their success in a way that is objec- 
tive, useful and acceptable to educators. 
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The following is a very brief summary of 
important points from this paper: 

• The cognitive perspective, which views 
learning as a complex interaction between 
the internal state of an individual and 
incoming information, is currently influen- 
tial in a variety of areas. The same set of 
cognitive principles applies to traditional 
environments as well as those that are com- 
puter-based and /or information-rich. The 
most effective learning environments: 

- consider the learner's pre-existing mental 
models 

- provide cues (such as strategies) for the cre- 
ation of new mental models 

- do not overtax the limitations of human 
attention 

- recognize the importance of context. 

• Technology is having a nnajor impact on our 
society and it is likely that technological 
familiarity (or lack thereoO will have pro- 
found impact on various groups and 
subgroups within our culture. We are 
becoming increasingly wealthy in 'informa- 
tion,' but it is not clear that this 
corresponds to an increase in our degree of 
understanding or meaningful use of that 
information. Educator:: must be aware of 
the impact of information technologies and 
structure clear rationales for their imple- 
mentation. 

• Educational technology research and evalu- 
ation must have a theoretical framework 
that incorporates what we know about 
learning as well as what we know about 
technology. Tlie effectiveness of any educa- 
tional innovation must be evaluated on the 
basis of its impact on learning (with learn- 
ing expectations clearly defined) in relation 
to the associated costs (time, money, and 
other). Research methodologies in educa- 
tion have not always led to acceptable 
results. 
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